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ABSTRACT 

In this theoretical paper, the questions of relevance and accountability in science 
education research are first reviewed and then addressed via comparison of the two most 
prevalent research methodologies: causal empirical-analytic, and naturalistic-hermeneutic. 
Critical theory and action research are proposed as alternative paradigms. A study planned 
at Purdue is discussed in terms of the three inteipretations.\ 

THE PROBLEM OF ACCOUNTABILITY IN SCIENCE EDUCATION RESEARCH 

There is a growing sense of frustration among members of the science education 
research community that results from the fact that the impact of research in science 
education on classroom practice has been disappointing, to say the least (Shymansky & 
Kyle, 1990). Wright (1993) offered four reasons why science education research is often 
irrelevant to the classroom teacher. 

1 . Science education research isn’t a well defined discipline supported by a 
sy.stematic base of knowledge. The preponderance of such research is carried out 
by graduate students as part of their Ph.D. program, rather than by seasoned team'-' 
with adequate funding. The preponderance of the research is “...conducted in 
isolation, with little focus, no agenda related to practice and no agreement on which 
methodologies are appropriate” 

2. Science education icseurchers have little or no interest in practical, applied 
classroom issues. There is a sense that researchers are mainly "...concerned with 
maintaining a status as the theoretical spokespersons for the enterprise. ... 
Researchers in science education want to be seen as having an elevated status rather 
than dirtying their hands with the day-to-day problems faced by teachers and 
students in the science classrooms..." 

3. ,Science education research hasn't recognized that educational practice isn't a 
scientific culture but a craft involving extensive unspoken personal experience and 
intuition. "Beliefs and practices drive the teaching enterprise, not research-based 
findings." Wright suggests that "...as long as we persist in applying the simplistic 
agricultural models of research (that look for simple cause and effeet) to the very 
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complex and contextual educational settings, then there is little hope of influencing 
the science teaching enterprise". 

4. Science education research is still in its infancy. We are still in the process of 
finding out what is out there in science classrooms. 

All but the last of Wright’s criticisms can be viewed as paradigmatic problems of 
science education research. Ph.D. research is principally done to demonstrate theoretical 
and methodological mastery within a paradigm, not to solve practical problems of 
instruction. The goal of this research isn’t an incremental improvement of practical 
instruction, but a revolutionary change in current practice as a result of instructional 
innovation (measured by quantitative methods) or a reconstruction of the learning 
experience and perspectives of the participants in instruction (mea.sured by qualitative 
means). 

Practicing teachers therefore view science education research as something apart 
from, theoretically superior to, and not really interested in classroom practice from the 
teacher's perspective, but from that of an external “expert.” They view science education 
research as something done to teachers and students by outsiders who aren’t interested in 
contributing to the everyday craft of teaching. Because this research doesn’t directly 
benefit the teacher, the teacher has little stake in trying to decipher and implement those 
pieces of research that are actually actionable. 

This dilemma concerning the relevance and validity of educational research isn't 
new; more than a half-century ago, John Dewey (1929) remarked on the direction and 
worth of educational research, and clearly subjugated theoretical rigor to educational 
practice: 

The answer is that ( 1 ) educational practices provide the data, the subject-matter, 
w'hich form the problems of inquiry... These educational practices are also (2) the 
final test of value of the conclusions of all researches... Actual activities in 
education test the worth of scientific results... They may be scientific in some other 
field, but not in education until they serve educational purposes, and whether they 
really serve or not can be found out only in practice, (p. 33) 

We would like to suggest that the irrelevancy of science education rese;uvh at present 
results from limitations in the two prevalent methodological paradigms used to conduct 
such research. These paradigms (quantitative, causal models of educational interventions 
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and qualitative, hermeneutic-natural approaches for the noninterventionary study of 
educational practice) are limited because neither fits actual instructional practice in the 
classroom. The methodologies and criteria used to judge worth in these kinds of research 
are driven by the search for theoretical perspectives sought by individuals who aren’t (at the 
moment) teachers, and these methodologies and criteria are both inappropriate for and 
insufficient to the needs of class oom teachers. 

Research Methodologies: The Quantitative Tradition 

The paradigm for quantitative research in science education has roots in tne natural 
sciences and psychology. Although there has been much debate regarding the evolution of 
the paradigmatic stances employed (positivism, neopositivism, behaviorism, and so forth), 
this research seeks causal relations between the kinds of instruction used and student 
learning. It frequently involves comparisons of an instructional innovation with “standard" 
instruction (interpreted as the absence of the innovation). The most striking examples of 
experimental design based on this paradigm are those of Campbell & Stanley (1963). 

In the wrong hands, this paradigm can give rise to a “sports mentality" approach to 
curriculum evaluation (Bodner, 1992) - limited comptu-isons of treatments and controls 
unreasonably removed from regular and possible classroom practice are statistically 
compared and victory or defeat for or against the innovation is declared. Because the 
methodology removes the experimental situation from the realm of the working classroom, 
assumes unreasonable controls of implementation and usually compares immature 
innovations, the results of these researcher-driven “horse races" often aren’t deemed 
w'orthwhile by working teachers. To them, the purpose of this research is to demonstrate 
researcher control over arcane experimental ritual, not to improve the lot of the teacher. 

Research Methodologies: The Qualitative Tradition 

Qualitative research provides a worthwhile paradigm to answer Wright's fourth 
criticism of research in science education - that we are still finding out what’s out there in 
the classroom. Unfortunately, as working professionals, teachers are quite aware of what 
is happening in their classes - they don’t believe that they need to be the subjects of 
anthropological research.' 

'In a recent seminar at a major university, a graduate student interested in the problems of a 
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The appeal of naturalistic research to working teachers is similar to that of Piaget’s 
theory of genetic epistemology. The information informs and provides material to reflect 
upon, but by nature isn’t designed to guide active intervention. Naturali.stic research 
methodologies have forsworn this role to the experimentalists. Unfortunately, teachers 
work in a world of continuous intervention into human learning. While the qualitative 
tradition in science education research can inform teachers, it is mainly done to inform 
re.searchers. Becau.se, once again, research doesn’t meet the needs of working teachers, it 
has little of any effect on classroom practice. 



Unorthodox Methodologies; Formative Research 

In what amou Us to a rejection of the “methodolitry” endemic to the qualitative and 
quantitative research paradigms, formative research has been conducted that was designed 
to benefit educational practices. This research is de.scribed by Walker (1992) as follows: 

"Formative researchers use such methods as reviewing research, con.sulting 
experts, constructing conceptual models, measuring characteristics of the intended 
audience for the educational program, and trying out prototypes in laboratories and 
in realistic field settings. They seek to learn about such matters as the readiness and 
needs of the audience, the value of the content to society and to the audience, the 
appeal of the planned program to the audience, the receptivity of teachers to it, and 
its utility and appeal for both students and teachers. Formative re.search is usually 
eclectic in its choice of techniques for eliciting data, including .self-reports (in the 
fonn of diaries, interviews or questionnaires), ob.servations, te.sts, and records" (p. 

1 11 ). 

Wall er describes the validity of this kind vif activity as follows: 

"...formative research draws its greate.st credibility from ( 1) the close similarity 
between the intended situation in which data arc collected and the situation of 
ultimate intere.st (trying out prototype materials in a classroom can be very close to 
using final versions in typical classrooms) and (2) the compelling face validity of 
the data collected (ob.servations of classroom interaction, test scores, and so on)" 

(p. 111). 



minority group iioteil that he received opposition to his classroom visits from members I'l 
the minority community - who were tired of being the subjects of another study of why 
they didn’t succeed. They wanteti someone to intervene, to help tliem becotne more 
successful. 
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He also cites examples of such unorthodox formative educational research. 

"Uri Treisman (Henkin and Treisman 1984; Treisman 1983), while a graduate 
student working with Professor Leon Henkin at the University of California at 
Berkeley, carried out a chain of studies that were by traditional standards 
methodologically primitive but nevertheless exceptionally productive". [...] "By 
any reasonable standards for curriculum research, this [Treisman' s study] was an 
outstanding study. The researcher focused his attention on the crucial practical 
problem, observed practices closely, kept himself open to a wide variety of 
evidence at every stage of the inquiry, compared circumstances in which a practice 
seemed to succeed with circumstances in which it failed, searched for factors in the 
situation that could be changed, redesigned practices to reflect what he thought he 
had learned from his observations, and tested the new practices by using the 
standards of achievement actually employed in the real course. His results have 
been widely reported and have already begun to influence research and practice in 
mathematics education (Gillman, 1990). And all this work was accomplished in 
three years on a modest budget". 



Dewey’s concerns with methodologically-rigorous research are recapitulated by Elliott 
Eisner ( 1979) when discussing formative research. As Walker notes ( 1992, p. 107) Eisner 
rejected the scientist's criterion of truth in favor of utility: “What we can productively ask of 
a set if ideas is not whether it is REALLY tme but whether it is useful.’’ 



To achieve profound insights into the active teaching and learning processes, 
formative researchers have deliberately chosen to reject methodological rigor in favor of 
utility. By the prevailing metl odological standards of the behavioral and social sciences, in 
contrast, these .studies are merely intriguing observations that prove nothing (Walker, 

1992, p. 1 14). 

However effective and relevant these cases of unconventional science educational research 
have been to working teachers, they don’t provide the guidance and methodological 
interpretation required to establish a systematic base of knowledge for science education 
research. Tt achieve this we need a paradigm. We suggest using Critical Theoiy 
(.Schroycr. 197.'^; Young. 1990) as the basis of that paradigm. 
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Critical theory was perhaps best set forth by the German sociologist-philosopher 
Jurgen Habermas (Gortzen & van Delder, 1970; McCarthy, 1979; Roderick, 1986; Sabia 
& Wallulis, 1983). Mezirow (1981, p. 3) describes Habermas as follows. 

"Jurgen Habermas is widely considered as the most influential thinker in Germany 
over the past decade [1970-80]. As a philosopher and sociologist he has mastered 
and creatively articulated an extraordinary range of specialized literature in the social 
sciences, social theory and the history of ideas i»' the provocative critical theory of 
knowledge and human interests. His roots are m the tradition of German thought 
from Kant to Marx, and he has been associa;ed with the Frankfurt School of critical 
theorists which pioneered in the study of th ; relationship of the ideas of Marx and 
Freud. " 



Habermas' Three Domains of Knowledge 

Habennas argued that human interest generates knowledge in three generic 
cognitive areas or domains shown in Table 1: technical ('vork), practical (social), and 
emancipatory. The.se areas are “knowledge constitutive ' because they determine 
appropriate actions for obtaining knowledge and judging v'hether knowledge claims can 
warranted. These areas define cognitive interests or learning domains, and are grounded in 
different aspects of social existence: work, interaction, and power. For our purposes, 
Habermas redefines human knowledge so that the mandate for formal knowledge creation 
is extended beyond the realm of the professional researcher and into that of all of the 
participants in human endeavors (e.g. teachers and students). 
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Table 1 

Habermas' Three Domains of Knowledg e (after Tinning. 1992) 



Type of Human Interest Kind of Knowledge Research Methods 



Technical - WORK Instrumental Positivistic Sciences 

(prediction) (causal explanation) (empirical-analytic methods) 



Practical - INTERACTION Practical Interpretive Research 

(interpretation and (understanding) (hermeneutic methods) 

understanding) 



Emancipatory - POWER Emancipation Critical Social Sciences 

(criticism and liberation) (reflection) Critical Theory Methods 

(including action research) 



Technical (Work) Knowledge 

Work, in its broadest sense, refers to the way one controls and manipulates one’s 
environment. Technical, or work, knowledge is commonly instrumental ;n nature - it is 
based on empirical observations and governed by technical rules. The criterion of effective 
control of reality directs what is (or isn’t) appropriate action for acquiring knowledge in this 
domain. The research methodology characterized by this domain is therefore the empirical- 
analytic .sciences using hypothetical-deductive theories. Habemias classifies much of what 
we consider physical science research domains - e.g. Physics, Chemistry and Engineering 
- as belonging to this domain. 



Practical (Social) Knowledge 

The domain ot practical or social knowledge is the result of human social interaction 
or “communicative action.” Social knowledge is govei ned by binding consensual norms, 
which define reciprocal expectations about behavior betw'ccn individuals. Social norms can 
be related to empirical or analytical propositions, but their validity is grounded “only m the 
intersuhjectivity of the mutual understanding of intentions.” The criterion of clarifcation of 
conditions for comnninication and intersubjectivity (the understimding of meaning: rather 
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than causality) is used to determine what is appropriate action for obtaining knowledge. 
Much of the historical-hermeneutic disciplines - descriptive social science, history, 
aesthetics, legal, ethnographic literary, and so forth are classified by Habermas as 
belonging to the domain of the practical. 

Emancipatory Knowledge 

The emancipatory domain includes “self-knowledge” or self-reflection. This 
ii. .olves interest in the way one's history and biography has expressed itself in the way one 
.sees oneself, one's roles, and social expectations. Emancipation occurs w'hen we are freed 
from libidinal, institutional, or environmental forces that limit our options and rational 
control over our lives that have b«:;en taken for granted as beyond human control. (A 
piocess also known as “reification.”) Insights gained through critical self-awareness are 
emancipatory in the sense that at least one can recognize the correct reasons for his or her 
problems. Knowledge is obtained by self-emancipation through reflection leading to a 
transformed consciousness or “perspective transfonnation.” According to Habermas 
(Mezirow, 1981), examples of critical sciences that o)x:rate within the emancipatory domain 
include psychoanalysis and the critique of ideologies (e.g,, sexual, racial, religious, 
educational, cKcupational, political, economic and technological ideologies), 

Habermas and Action Research 

Although action research will be described in detail in the next section, if we are to 
presume that critical theory provides a paradigm for action research methodologies, it is 
important to establish the relationship between action research and critical theory. The 
educational action research movement predates Habeimas. Action research can readily he 
traced back to the end of World War II when social psychologist Kurt Lewin (1946) 
(Icvelcped most of the current methodological characteristics. Habermas has provided a 
the '•etical background to the methodologies advocated by proponents of action research 
(Kemmis & McTaggart, 1990), not vice-versa. 

Kemmis and McTaggart ( 1990) point out that there is considerable “...ilebate about 
the extent to which action research is a research nietluHlolof’x or teelwUjiie on one hand or 
a hroaJ a[)[)roa('h to social research and reform on the other." Kemmis also raises the 
issue of where action research should be kKatcii, either as "...part of the ider field r>f 
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social theory or in the narrower focus of education and the development of educational 
theory.” This is exemplified by a basic dichotomy between the different schools of action 
research, where some researcher are concerned with ‘‘...the development of teacher's (or 
others') theories of education and society while other researcher.! concern themselves with 
questions of social and educational change - improvement, reform and innovation.” 

ACTION RESEARCH 

Action research (Kemmis, 1983; McTaggart, 1991; Tinning, 1992) has been 
described as a recursive, reflexive, dialectical technique that aims to help people investigate 
reality in order to change it, or to change reality in order to investigate it. by changing their 
practices in a collaborative, self-reflective spiral of cycles (Kemmis and McTaggart, 1988). 
It is an informal, qualitative, formative, subjective, interpretive, reflective, and experiential 
model of inquiry in which all individuals involved in the study are knowing and 
contributing participants (Hopkins, 1985). Action research is a deliberate social process 
designed to help members of a group learn more about tlieir practices, their knowledge of 
their practices, the social structures that constrain their practices, and the social environment 
in which these practices are expressed and realized. A ”... process of learning by doing - 
and leiu-ning with others by changing the ways they interact in a shared social world in 
which, for better or for worse, we live with the consequences of one another’s actions” 
(Kemmis and McTaggart, 1988). 

Action Research represents a growing field of educational research whose chief 
identifying characteristic is the recognition of the pragmatic requirements of educational 
practitioners for organized reflective inquiry into classroom instruction. It is a process 
designed to empower all participants in the educational process (students, instructors and 
other parties) with the means to improve the practices conducted within the educational 
experience. All participants are knowing, active members of the research process. 

The action research framework is most appropriate lor participants who recognize 
the existence of shortcomings in their educational activities and who would like to adopt 
some initial stance in regard to the problem, formulate a plan, cany out an intervention, 
evaluate the outcomes and develop further strategies in an iterative fa.shion. given the initial 
stance that pedagogical shortcomings exist. The action research tradition allows the use of 
many techniques of data collection, and action research studies typically make use of both 
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qualitative and quantitative measures and analyses. These data are used to suggest and 
refine these interventions in the curricular activities and to guide further research during 
succeeding terms. 

Some of the most widely accepted definitions of action research include the following: 

[Action research] ...aims to contribute both to the practical concerns of people in an 
immediate problematic situation and to the goals of social science by joint 
collaboration within a mutually acceptable ethical framework. (Rapoport, 1970) 

Action Research is a form of self-reflective inquiry undertaken by participants in 
social (including educational) situations in order to improve the rationality and 
Justice of (a) their own social or educational practices, (b) their understanding of 
these practices, and (c) the situations in which the practices are carried out. It is 
most rationally empowering when undertaken by participants 
collaboratively... sometimes in cooperation with outsiders. (Kemmis, 1983) 

[Action research] ...is the systematic study of attempts to improve educational 
practice by groups of participants by means of their own practical actions and by 
means of their own reflection upon the effects of those actions. (Ebbut, 1983) 

In short, action research is characterized by the constraints and strengths that provide a 
re> irch methodology intended to be a workable technique for working classroom teachers. 



Action Research Designs 

The essentials of action research design are described by Elliott (1978) in terms of the 
following characteristic cycle: 

• initially an exploratory stance is adopted, where an understanding of a problem 
is develop)ed and plans are made for some form of interventionary strategy. 
(The Reeonnuissume eft General Plan) 

• then the intervention is carried out. (The Action in Action Research) 

• during and around the time of the intervention, pertinent observations arc 
collected in various forms. (Monitorinf> the Implementation) 

• the data are examined for trends and characteristics, and a new strategy 
developed for implementation. (The Revised Plan or Maintaininf> the Action) 

• the new interventional strategies are carried out. and the cyclic process repeats, 
continuing until a sufficient understanding of (or implementable solution for) 
th: problem is achieved. 
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Action resetirch is iterative or cyclical in nature and is intended to foster deeper 
understanding of a given situation, starting with conceptualizing and particularizing the 
problem and moving through several interventions and evaluations. One representation of 
an action research protocol by Kemmis (1983) is provided in Figure 1, which displays the 
iterative nature of action research along with the major steps of planning, action, 
observation and reflection before revising the plan. In some ways, action research is 
similar in nature to the numerical technique know'n as successive approximation - the goal 
is to achieve a desirable outcome by a process of repeated iterations. 



CYCLE 1 



CYCLE 2 




Fi)i'Uv I.. Action icscaich protocol (after Kemmis. in Hopkins. lUO.t) 



O 

ERIC 
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Later cycles through the action research spiral reflect changes made as a result of 
reflection on the experiences of earlier iterations. Figure 2, for example, reflects the 
evolution of the general idea or main topic of interest throughout the process. 



ID Initial 
Idea 

" X 

ReconnaissanGe 

I 

Geneial Plan 



Action Steps 1 



Action Steps 2 



Action Steps 3 



Implement 
Action Steps 1 



Monitor Implementation 
and Effects 
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E 
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Implement Next 

“*■ ’ Action Steps 

7 
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and Effects 
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E 
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Action Steps 3 
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Elliott's model emphasizes constant evolution and redefinition of the original goal 
through a series of reconnaissances recurring every cycle. The reconnaissance necessarily 
includes some degree of analysis. This design permits much greater flexibility, and seeks 
to “...recapture some of the ‘messiness’ which the Kemmis version tends to gloss” [over] 
(Hopkins, 1985). Ebbutt ( 1983) illustrates the evolution of the through the 

spiral analogy in Figure 3. 




etc 



Figure 3 . Action research protocol (after Ebbutt. in Hopkins. 1993) 

The Role of Comtnunication 

One of the primary' characteristics of action research is the degree of empowerment given to 
all participants. Involvement i'^ of a knowing nature, with no hidden controls or 
preemption of direction by the researcher. All participants negotiate meaning from the data 
and contribute to the selection of interventionary strategies, including the researchers, the 
teachers, and the students. 
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Elliott considers the need for communication between all participants to be of 
paramount importance: 

Since action research looks at a problem from the point of view of those involved it 
can only be validated in unconstrained dialogue with them. 

Since action research involves unconstrained dialogue between “re.searcher" 

(whether he be an outsider or teacher/researcher) and the participants, there must be 
free information flow between them. (Elliott, 1978) 

Reflection 

Perhaps the key component involved in action research is the notion of praxis. 
Action research is intended to be the “...reflective counterpart of practical diagnosis ..." 
(Elliott, 1978). Schon describes the use of reflection to generate models from a body of 
previous knowledge. The.se models are used to frame a problem; then experiments are 
performed to bring about outcomes which arc subjected to further analysis. This model 
(called reflection-in-action) frames means and ends interdepcndcntly and recognizes that 
there is little or no separation of research from practice, little or no separation of knowing 
and doing (Schon, 1983). Schbn's model of reflection-in-action compliments the iterative 
and investigative natures of action research. 

AN EXAMPLE OF THE THREE METHODOLOGIES 

The design of an investigation perfomied at Purdue University might provide an 
ideal example of the difference between the approaches taken by someone guided by the 
three research methodologies (quantitative-experimental, qualitative-naturalistic, and action 
research). 

Students enrolled in the organic chemistry course taught within the Pharmacy 
Department at Purdue were divided into two groups, more or less randomly. One group 
experienced a classic lecture approach to organic cheniisti^ by two instructors with many 
years of experience at teaching organic chemistry via well-crafted lectures. The other group 
experienced a class in which the instructor asked them to work in groups of three on 
problems he brought to class. After the students worked together in class, they shared their 
results with the instructor, who then tried to examine the logical consequences of their 
answers. The students were encouraged to work together outside the classroom. The 
groups will also provided witli time to discuss each hour exam before they split up to write 
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individual responses to the hour-exam questions. 

This “experiment" was done at the request of the instructor of the experimental 
section, who believed that having students work together in groups would increase their 
retention of the material learned in the first half of this two-semester organic course. As 
the reader might expect, we collected quantitative data on student performance. The goal of 
this phase of the research, which was guided by the quantitative paradigm, was to probe 
questions such as the following. 

Is there any difference on the hour exams between the two sections? 

Is there an increase in the retention of material between semesters of the organic 

sequence? 

Data designed to guide articulation of this approach to teaching organic chemistry in 
the future was collected by qualitative, ethnographic techniques taken from the naturalistic 
research tradition. On the basis of field notes, which report what happened on a day-to-day 
basis in the classroom, observations of both the students and their instructor during the 
course of the semester, and interviews with both students and the instructor, the research 
team hoped to find answers to the following kinds of questions: 

How do students interact when they work together? 

What do the students choose to talk about? 

What do the students discuss when they preview exams'.^ 

What are the students perceptions of their instructor and each other? 

What are the instructor's and the students' beliefs about how learning occurs? 

What do the students believe is the role of the instructor in organic chemistry ? 

What do students understand about organic chemistry'.^ 
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The collection of the most important (and what we believe to be the most 
informative) data for the instructor was guided by critical theory and action research 
methodology. The goal of this phase of the research project was to collect information 
regarding our intervention, and try' to judge its’ worth with the active, knowing 
participation of the students involved, in order to guide further interventions in instruction. 
The kinds of critical questions asked included: 

What is the nature of the dissatisfaction that led the instructor to change the 
instructional situation? 

What factors make it difficult to change the instructional situation? 

What factors interfere with the ease with which this technique can be used by other 
instructors, or transported to different institutions? 

What effect does this mode of instruction have on the instructor’s attitude toward 
teaching? 

Is this approach to organic chemistry more fun to teach? 

What effect does this have on students perception of the difficulty of organic 
chemistry? 

What do students understand about organic chemistry? 

Does this change when we alter the approach taken to instruction? 

Does the new instructional mode produce students who think more like an organic 
chemist? 

Does it produce students who understand what organic chemists do? 



The goal of this research differs significantly from the goals of many experiments 
designed to probe the effect of alternative approaches to instruction. Our goal wasn't to 
decide whether this technique is “better" than the tradhional approach to teaching organic 
chemistry. It was to obtain infomiation that would allow us to maximize whatever 
beneficial effect this approach to instruction provides, and to minimize any negative effects 
that arise. By changing the nature of the questions being asked during this research, we 
expected to obtain information that is more useful not only to those doing the research, hut 
also to the instructor who teaches the course. Ultimately, the relevance of our research is to 
be judged by the participants - the instructor and the students, rather than the profession 
science ethication research community. 
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